Harnessing Models of Users' Goals to Mediate Clarification Dialog in Spoken Language Systems

Speaker-independent speech recognition systems are being used with increasing frequency for command and control applications. To date, users of such systems must contend with the fragility of recognition with subtle changes in language usage and environmental acoustics. We describe work on coupling speech recognition systems with temporal probabilistic user models that provide inferences about the intentions associated with utterances. The methods can be employed to enhance the robustness of speech recognition systems by endowing the systems with an ability to reason about the costs and benefits of action in a setting and to make decisions about the best action to take given uncertainty about the meaning behind acoustic signals. The methods have been implemented in the form of a dialog clarification module that can be integrated with legacy spoken command and control systems. We describe representation and inference procedures and present details on the operation of an implemented spoken command and control development environment named DeepListener.